视频实例分割旨在预测每个帧的对象分割掩码，并关联多个帧的实例。最近的端到端视频实例分割方法能够在直接并行序列解码/预测框架中共同执行对象分割和实例关联。尽管这些方法通常可以预测较高质量的对象分割掩码，但它们可能无法在具有挑战性的情况下与实例相关联，因为它们没有明确对相邻帧的时间实例一致性进行建模。我们提出了一个一致的端到端视频实例分割框架，并在框架间反复注意，以建模相邻帧的时间实例一致性和全局时间上下文。我们的广泛实验表明，框架间的重复注意显着提高了时间实例的一致性，同时保持对象分割掩模的质量。我们的模型在YouTubevis-2019（62.1 \％）和YouTubevis-2021（54.7 \％）数据集上都达到了最新的精度。此外，定量和定性结果表明，所提出的方法可以预测更具时间一致的实例分割掩码。

未经监督的域名自适应人员重新识别（Reid）已被广泛调查以减轻域间隙的不利影响。这些作品假设目标域数据可以一次访问。然而，对于真实世界的流数据，这会阻碍及时适应改变数据统计数据以及对增加样本的充分利用。在本文中，为了解决更实际的情况，我们提出了一项新任务，终身无监督域自适应（Luda）人Reid。这是具有挑战性的，因为它要求模型不断适应目标环境的未标记数据，同时减轻灾难性的遗忘，为这么细粒度的检索任务。我们为这项任务设计了一个有效的计划，被称为Cluda-Reid，在那里反忘记与适应协调。具体地，提出了基于元的协调数据重放策略来重播旧数据并以协调的优化方向更新网络，以便适应和记忆。此外，我们提出了符合基于检索的任务的目标的旧知识蒸馏/继承的关系一致性学习。我们设置了两个评估设置来模拟实际应用方案。广泛的实验展示了我们Cluda-Reid与具有动态目标流的静止目标流和场景的方案的有效性。

多摄像机跟踪系统在需要高质量跟踪结果的应用中获得普及，例如摩擦结账，因为单眼多物体跟踪（MOT）系统由于闭塞而在杂乱和拥挤的环境中经常失败。通过恢复部分3D信息，多个高度重叠的相机可以显着减轻问题。但是，使用不同的相机设置和背景创建高质量多摄像头跟踪数据集的成本在该域中的数据集比例限制了数据集尺度。在本文中，我们在自动注释系统的帮助下提供了五种不同环境的大型密集标记的多摄像头跟踪数据集。该系统使用重叠和校准的深度和RGB相机来构建高性能3D跟踪器，可自动生成3D跟踪结果。使用摄像机参数将3D跟踪结果投影到每个RGB摄像头视图以创建2D跟踪结果。然后，我们手动检查并更正3D跟踪结果以确保标签质量，比完全手动注释便宜得多。我们使用两个实时多相机跟踪器和具有不同设置的人重新识别（REID）模型进行了广泛的实验。该数据集在杂乱和拥挤的环境中提供了更可靠的多摄像头，多目标跟踪系统的基准。此外，我们的结果表明，在此数据集中调整跟踪器和REID模型显着提高了它们的性能。我们的数据集将在接受这项工作后公开发布。

New architecture GPUs like A100 are now equipped with multi-instance GPU (MIG) technology, which allows the GPU to be partitioned into multiple small, isolated instances. This technology provides more flexibility for users to support both deep learning training and inference workloads, but efficiently utilizing it can still be challenging. The vision of this paper is to provide a more comprehensive and practical benchmark study for MIG in order to eliminate the need for tedious manual benchmarking and tuning efforts. To achieve this vision, the paper presents MIGPerf, an open-source tool that streamlines the benchmark study for MIG. Using MIGPerf, the authors conduct a series of experiments, including deep learning training and inference characterization on MIG, GPU sharing characterization, and framework compatibility with MIG. The results of these experiments provide new insights and guidance for users to effectively employ MIG, and lay the foundation for further research on the orchestration of hybrid training and inference workloads on MIGs. The code and results are released on https://github.com/MLSysOps/MIGProfiler. This work is still in progress and more results will be published soon.

Learning to predict masked tokens in a sequence has been shown to be a powerful pretraining objective for large-scale language models. After training, such masked language models can provide distributions of tokens conditioned on bidirectional context. In this short draft, we show that such bidirectional conditionals often demonstrate considerable inconsistencies, i.e., they can not be derived from a coherent joint distribution when considered together. We empirically quantify such inconsistencies in the simple scenario of bigrams for two common styles of masked language models: T5-style and BERT-style. For example, we show that T5 models often confuse its own preference regarding two similar bigrams. Such inconsistencies may represent a theoretical pitfall for the research work on sampling sequences based on the bidirectional conditionals learned by BERT-style MLMs. This phenomenon also means that T5-style MLMs capable of infilling will generate discrepant results depending on how much masking is given, which may represent a particular trust issue.

This paper presents a practical global optimization algorithm for the K-center clustering problem, which aims to select K samples as the cluster centers to minimize the maximum within-cluster distance. This algorithm is based on a reduced-space branch and bound scheme and guarantees convergence to the global optimum in a finite number of steps by only branching on the regions of centers. To improve efficiency, we have designed a two-stage decomposable lower bound, the solution of which can be derived in a closed form. In addition, we also propose several acceleration techniques to narrow down the region of centers, including bounds tightening, sample reduction, and parallelization. Extensive studies on synthetic and real-world datasets have demonstrated that our algorithm can solve the K-center problems to global optimal within 4 hours for ten million samples in the serial mode and one billion samples in the parallel mode. Moreover, compared with the state-of-the-art heuristic methods, the global optimum obtained by our algorithm can averagely reduce the objective function by 25.8% on all the synthetic and real-world datasets.

Through a study of multi-gas mixture datasets, we show that in multi-component spectral analysis, the number of functional or non-functional principal components required to retain the essential information is the same as the number of independent constituents in the mixture set. Due to the mutual in-dependency among different gas molecules, near one-to-one projection from the principal component to the mixture constituent can be established, leading to a significant simplification of spectral quantification. Further, with the knowledge of the molar extinction coefficients of each constituent, a complete principal component set can be extracted from the coefficients directly, and few to none training samples are required for the learning model. Compared to other approaches, the proposed methods provide fast and accurate spectral quantification solutions with a small memory size needed.

Neural operators, which emerge as implicit solution operators of hidden governing equations, have recently become popular tools for learning responses of complex real-world physical systems. Nevertheless, the majority of neural operator applications has thus far been data-driven, which neglects the intrinsic preservation of fundamental physical laws in data. In this paper, we introduce a novel integral neural operator architecture, to learn physical models with fundamental conservation laws automatically guaranteed. In particular, by replacing the frame-dependent position information with its invariant counterpart in the kernel space, the proposed neural operator is by design translation- and rotation-invariant, and consequently abides by the conservation laws of linear and angular momentums. As applications, we demonstrate the expressivity and efficacy of our model in learning complex material behaviors from both synthetic and experimental datasets, and show that, by automatically satisfying these essential physical laws, our learned neural operator is not only generalizable in handling translated and rotated datasets, but also achieves state-of-the-art accuracy and efficiency as compared to baseline neural operator models.

Diagram object detection is the key basis of practical applications such as textbook question answering. Because the diagram mainly consists of simple lines and color blocks, its visual features are sparser than those of natural images. In addition, diagrams usually express diverse knowledge, in which there are many low-frequency object categories in diagrams. These lead to the fact that traditional data-driven detection model is not suitable for diagrams. In this work, we propose a gestalt-perception transformer model for diagram object detection, which is based on an encoder-decoder architecture. Gestalt perception contains a series of laws to explain human perception, that the human visual system tends to perceive patches in an image that are similar, close or connected without abrupt directional changes as a perceptual whole object. Inspired by these thoughts, we build a gestalt-perception graph in transformer encoder, which is composed of diagram patches as nodes and the relationships between patches as edges. This graph aims to group these patches into objects via laws of similarity, proximity, and smoothness implied in these edges, so that the meaningful objects can be effectively detected. The experimental results demonstrate that the proposed GPTR achieves the best results in the diagram object detection task. Our model also obtains comparable results over the competitors in natural image object detection.

Despite excellent performance in image generation, Generative Adversarial Networks (GANs) are notorious for its requirements of enormous storage and intensive computation. As an awesome ''performance maker'', knowledge distillation is demonstrated to be particularly efficacious in exploring low-priced GANs. In this paper, we investigate the irreplaceability of teacher discriminator and present an inventive discriminator-cooperated distillation, abbreviated as DCD, towards refining better feature maps from the generator. In contrast to conventional pixel-to-pixel match methods in feature map distillation, our DCD utilizes teacher discriminator as a transformation to drive intermediate results of the student generator to be perceptually close to corresponding outputs of the teacher generator. Furthermore, in order to mitigate mode collapse in GAN compression, we construct a collaborative adversarial training paradigm where the teacher discriminator is from scratch established to co-train with student generator in company with our DCD. Our DCD shows superior results compared with existing GAN compression methods. For instance, after reducing over 40x MACs and 80x parameters of CycleGAN, we well decrease FID metric from 61.53 to 48.24 while the current SoTA method merely has 51.92. This work's source code has been made accessible at https://github.com/poopit/DCD-official.